Marsabit County
Retrieval-Augmented Clinical Benchmarking for Contextual Model Testing in Kenyan Primary Care: A Methodology Paper
Mutisya, Fred, Gitau, Shikoh, Syovata, Christine, Oigara, Diana, Matende, Ibrahim, Aden, Muna, Ali, Munira, Nyotu, Ryan, Marion, Diana, Nyangena, Job, Ongoma, Nasubo, Mbae, Keith, Wamicha, Elizabeth, Mibuari, Eric, Nsengemana, Jean Philbert, Chidede, Talkmore
Large Language Models (LLMs) hold promise for improving healthcare access in low-resource settings, but their effectiveness in African primary care contexts remains under-explored. We present a rigorous methodology for creating a benchmark dataset and evaluation framework focused on Kenyan Level 2-3 (dispensary and health center) clinical care. Our approach leverages retrieval-augmented generation (RAG) to ground questions and answers in Kenya's national clinical guidelines, ensuring content aligns with local standard-of-care. The guidelines were digitised, chunked, and indexed for efficient semantic retrieval. Gemini Flash 2.0 Lite was then prompted with relevant guideline excerpts to generate realistic clinical questions, multiple - choice answers, and reasoning scenarios with source citations in English and Swahili. We engaged Kenyan physicians in a co - creation process to refine the dataset's relevance and fairness, and instituted a blinded expert validation pipeline to review for clinical accuracy, clarity, and cultural appropriateness. The resulting Alama Health QA dataset comprises thousands of regulator-aligned question-answer pairs spanning common outpatient conditions in English and Swahili. Beyond standard accuracy metrics, we propose innovative evaluation measures targeting clinical reasoning, safety, and adaptability (e.g. Initial results highlight significant performance gaps in state - of-the - art LLMs when confronted with localized scenarios, echoing recent findings that LLM accuracy on African medical questions lags behind performance on U.S. benchmarks. Our work demonstrates a pathway for dynamic, locally-grounded benchmarks that can evolve with guidelines, providing a crucial tool for safe and effective deployment of AI in African healthcare. Advances in large language models have spurred interest in their potential to augment medical services, especially in low-and middle -income countries facing clinician shortages(Bekbolatova et al., 2024). By handling routine queries or providing decision support, LLMs might help bridge gaps in healthcare access across Africa.
- Africa > South Africa > Gauteng > Johannesburg (0.04)
- Africa > Kenya > Kisumu County > Kisumu (0.04)
- Europe > United Kingdom (0.04)
- (3 more...)
Improving the accuracy of food security predictions by integrating conflict data
Bertetti, Marco, Agnolucci, Paolo, Calzadilla, Alvaro, Capra, Licia
Food security (FS) is a complex and multifaceted problem, influenced by several factors such as weather events, economic shocks, and natural disasters. Understanding the dynamics of food security is crucial for effective policymaking and humanitarian efforts. While conflicts and violent events increasingly stand out as key drivers of food crises[1], the depth of their impact remains largely underexplored. Examining the quantitative aspects of this impact is essential for developing more targeted interventions and strategies to address the complex interplay between conflict and food security. Existing research tends to be qualitative in nature (Kemmerling et al.2022; Brown et al. 2020; Brown et al. 2021), leaving a significant gap in understanding the quantitative aspects of how conflicts impact FS levels. By delving into quantitative analyses, we can not only enhance our comprehension of the magnitude of the problem but also pave the way for evidence-based decision-making in efforts to alleviate food insecurity in conflict-affected regions. Regarding the qualitative study of conflicts and FS, Kemmerling et al.(2022)[2] provided a comprehensive explanation on how violence and armed conflicts impact FS through destruction, displacement, financing of conflicts and food being used as a weapon. The authors call for better conflict data collection, and an increase in focus on the study of conflicts early warnings.
- North America > United States (0.93)
- Africa > Ethiopia (0.29)
- Asia > Russia (0.28)
- (12 more...)
BART-SIMP: a novel framework for flexible spatial covariate modeling and prediction using Bayesian additive regression trees
Jiang, Alex Ziyu, Wakefield, Jon
Prediction is a classic challenge in spatial statistics and the inclusion of spatial covariates can greatly improve predictive performance when incorporated into a model with latent spatial effects. It is desirable to develop flexible regression models that allow for nonlinearities and interactions in the covariate structure. Machine learning models have been suggested in the spatial context, allowing for spatial dependence in the residuals, but fail to provide reliable uncertainty estimates. In this paper, we investigate a novel combination of a Gaussian process spatial model and a Bayesian Additive Regression Tree (BART) model. The computational burden of the approach is reduced by combining Markov chain Monte Carlo (MCMC) with the Integrated Nested Laplace Approximation (INLA) technique. We study the performance of the method via simulations and use the model to predict anthropometric responses, collected via household cluster samples in Kenya.
- North America > United States (0.46)
- Africa > Kenya > Nairobi City County > Nairobi (0.04)
- Africa > Kenya > Mombasa County > Mombasa (0.04)
- (25 more...)
- Research Report > New Finding (0.46)
- Research Report > Experimental Study (0.46)
A mixed model approach to drought prediction using artificial neural networks: Case of an operational drought monitoring environment
Adede, Chrisgone, Oboko, Robert, Wagacha, Peter, Atzberger, Clement
Droughts, with their increasing frequency of occurrence, continue to negatively affect livelihoods and elements at risk. For example, the 2011 in drought in east Africa has caused massive losses document to have cost the Kenyan economy over $12bn. With the foregoing, the demand for ex-ante drought monitoring systems is ever-increasing. The study uses 10 precipitation and vegetation variables that are lagged over 1, 2 and 3-month time-steps to predict drought situations. In the model space search for the most predictive artificial neural network (ANN) model, as opposed to the traditional greedy search for the most predictive variables, we use the General Additive Model (GAM) approach. Together with a set of assumptions, we thereby reduce the cardinality of the space of models. Even though we build a total of 102 GAM models, only 21 have R2 greater than 0.7 and are thus subjected to the ANN process. The ANN process itself uses the brute-force approach that automatically partitions the training data into 10 sub-samples, builds the ANN models in these samples and evaluates their performance using multiple metrics. The results show the superiority of 1-month lag of the variables as compared to longer time lags of 2 and 3 months. The champion ANN model recorded an R2 of 0.78 in model testing using the out-of-sample data. This illustrates its ability to be a good predictor of drought situations 1-month ahead. Investigated as a classifier, the champion has a modest accuracy of 66% and a multi-class area under the ROC curve (AUROC) of 89.99%
- Africa > East Africa (0.34)
- Europe > Austria > Vienna (0.14)
- Africa > Kenya > Wajir County > Wajir (0.05)
- (13 more...)